Performance Evaluation with Transactions Data: The Stock Selection of Investment Newsletters

This paper analyzes the equity-portfolio recommendations made by investment newsletters. Overall, there is no significant evidence of superior stock-picking ability for this sample of 153 newsletters. Moreover, there is no evidence of abnormal short-run performance persistence ("hot hands"). The comprehensive and bias-free transactions database also allows for insights into the precision of performance evaluation. Using a measure of precision defined in the paper, a transactions-based approach yields a median improvement of 10percent over a corresponding factor model. This compares favorably with the precision gained by adding factors to the CAPM. INVESTMENT NEWSLETTERS HAVE EXISTED since a t least the early part of this century, and the current industry of more than 500 active letters has approximately 2 million subscribers (Hulbert (1996)). The typical newsletter is produced by a small staff and provides a wide range of advice targeted at the retail investor. Successful publications can earn millions of dollars in subscription revenue and allow their editors to become frequent speakers a t investment seminars (Brimelow (1986)). This paper undertakes a detailed study of newsletters' equity recommendations. The dataset spans 17 years, is free of survivor and back-fill biases, and contains every recommended (long) transaction for 153 newsletters. These data allow for an analysis of two questions. First, do investment newsletters have stock-selection ability? Second, can transactions data be used to improve the precision of performance evaluation? The Hulbert Financial Digest (HFD) has been tracking investment-newsletter recommendations since 1980. The only previous papers to use the HFD database are those by Graham and Harvey (1996, 1997) and Graham (1998), which focus on newsletters7 timing between stocks and cash. The present paper is the first attempt to study the specific equity recommendations of " Department of Economics, Harvard University and NBER. I thank John Campbell, Gary Chamberlain, Paul Gompers, John Graham, Campbell Harvey, Leslie Jeng, Susie Metrick, Jack Porter, Andrei Shleifer, Rene Stultz, Sheridan Titman, Richard Zeckhauser, an anonymous referee, and semiliar participants at the NBER, the 1997 NBER Summer Institute, Harvard, MIT, and Chicago for helpful comments, Daniel Chen and Alex Tsai for editorial assistance, and James Choi and Tom Knox for excellent and tireless research assistance. Mark Hulbert was very generous with his time, advice, and data. This project was partially supported by the Richard Herrnstein Fund at Harvard University. 1744 The Journal of Finance these financial advis0rs.l In this respect, the paper is similar to other studies of "expert" equity recommendations such as those by Barber and Loeffler (1993) (on The Wall Street Journal's Dartboard column), Desai and Jain (1995) (on "Superstar" money managers in Barren's), and Womack (1996) (on brokerage analysts). To carry out the analysis, the paper uses performanceevaluation methods developed in several recent studies of mutual funds (Malkiel (1995), Carhart (1997a), and Daniel, Grinblatt, Titman, and Wermers (DGTW) (1997)). Using these methods, I find no evidence of abnormal stockselection ability in this sample of newsletters. In addition to determining the investment value of newsletters' stock selection advice, this paper also attempts to shed light on the power and limitations of various performance-evaluation methodologies. The transactions-level detail and bias-free construction of the HFD database allow for a comparison of results from several different methods and insight into their relative precisions. Though such comparisons can also be carried out on simulated data, such tests may miss crucial elements of actual managed portfolios and can lead to biased results. The HFD database provides a rare natural experiment. Using a measure of "precision" defined in the paper, I find that the transactions-based approach of DGTW (1997) yields a median improvement of 10 percent over the 4-factor model of Carhart (1997a), with the former approach providing more precise estimates of abnormal performance for more than 80 percent of the newsletters. This compares with a median improvement of less than 1percent for the 4-factor model over the CAPM. These results have useful implications given the advent of transactions and holdings databases in studies of insider trading (Eckbo and Smith (1998), Jeng (1998)), of individual investors (Barber and Odean (1999), Grinblatt and Keloharju (1998)), and of mutual funds (DGTW (1997)). Section I of the paper describes the HFD database in detail and provides summary statistics on newsletter recommendations and performance. Section I1 contains the analysis of stock-selection performance. Section I11 analyzes the consistency and relative precision of the three performance-evaluation models. Section IV looks for short-term persistence, or "hot hands," in newsletter stock-selection ability. Section V concludes the paper. Two appendixes supplement the text: Appendix A discusses the calculation of newsletter returns and Appendix B describes the construction of the return series used in the DGTW (1997) characteristic-matching model.